Taking It to the Streets: How Virtual Reality Can Change Mobile Computing

Virtual reality has long been an indoor affair. Whether constrained by stationary computers or displays, or by the limitations of our tracking technologies, researchers typically build virtual environments that work within a single physical room or a portion of a room. Even distributed virtual reality systems usually interconnect two or more such indoor spaces. Meanwhile, as computers grow ever smaller and faster, mobile computing is becoming an increasingly important part of our daily lives, accompanying us wherever we go, outdoors, as well as indoors. What will it take for virtual reality to move outdoors and finally see the light of day? And, why should we care? Within the virtual reality research community, work on augmented reality has already begun to explore outdoor environments—tracking using computer vision, gyroscopes, accelerometers, compasses, and GPS; and experimenting with (barely) wearable testbeds. I will discuss why virtual reality (especially in the form of augmented reality) and mobile computing are a synergistic combination, and will provide an overview of the research problems that must be addressed for mobile augmented reality systems to play a major role in our future. Among the issues that I will review are overcoming physical and aesthetic barriers to mobility and wearability; tracking and registration of heads, hands, bodies, and other objects; rendering virtual objects in the real world; and developing sufficiently high quality displays. Equally important is the design of head-tracked user interfaces that are well suited to mobility. Wearable systems will need to support collaboration among mobile users, facile interaction with real and virtual objects, and coordination across a wide range of heterogeneous displays and devices. Key here is the volatile nature of mobile interactions—users continually move into and out of the presence of other users, devices, and objects, and rapidly change tasks. Furthermore, augmented reality makes it possible for real and virtual objects to share the same display space, creating the potential for a variety of visually confusing relationships as objects overlap and occlude each other. Avoiding these problems will require that the virtual world be redesigned and laid out on the fly, to maintain desired visual relationships between virtual objects and other real and virtual objects. Steven Feiner (http://www.cs.columbia.edu/~feiner) is a Professor of Computer Science at Columbia University, where he directs the Computer Graphics and User Interfaces Laboratory (http://www.cs.columbia.edu/graphics/). He received a Ph.D. in Computer Science from Brown University in 1987. Prof. Feiner’s research interests include virtual environments and augmented reality, knowledge-based design of graphics and multimedia, wearable computing, information Proceedings of the IEEE Virtual Reality 2003 (VR’03) 1087-8270/03 $17.00 © 2003 IEEE visualization, and hypermedia. He is coauthor of Computer Graphics: Principles and Practice (Addison-Wesley, 1990) and of Introduction to Computer Graphics (Addison-Wesley, 1993). He is an associate editor of ACM Transactions on Graphics, has served on the executive boards of the IEEE Computer Society Technical Committee on Visualization and Graphics and the IEEE Computer Society Task Force on Human-Centered Information Systems, and is a member of the steering committees for the IEEE Symposium on Information Visualization, the IEEE and ACM International Symposium on Mixed and Augmented Reality, and the IEEE Computer Society Technical Committee on Wearable Information Systems. Prof. Feiner is program co-chair for the 2003 IEEE International Symposium on Wearable Computers. Over the past few years, he has been general chair of IEEE Information Visualization 2001, symposium co-chair of the 2001 IEEE and ACM International Symposium on Augmented Reality, program co-chair of the 2001 International Symposium on Mixed Reality, and program cochair of IEEE Virtual Reality 2000. In 1991 he received an Office of Naval Research Young Investigator Award. Proceedings of the IEEE Virtual Reality 2003 (VR’03) 1087-8270/03 $17.00 © 2003 IEEE